1. Statistics for the Social Sciences
Psychology 340
Spring 2005
Effect sizes & Statistical Power

2. Outline
Effect size: Cohen’s d
Error types
Statistical Power Analysis

3. Performing your statistical test
There really isn’t an effect
Real world (‘truth’)
There really is
an effect
H0 is correct
H0 is wrong
Reject H0
Experimenter’s conclusions
Fail to Reject H0

4. Performing your statistical test
Real world (‘truth’)
H0 is correct
H0 is wrong

5. Performing your statistical test
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
Real world (‘truth’)
H0 is correct
H0 is wrong
So there is only one distribution
So there are two distributions
The original (null) distribution
The new (treatment) distribution
The original (null) distribution

6. Performing your statistical test
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
Real world (‘truth’)
H0 is correct
H0 is wrong
So there is only one distribution
So there are two distributions
The original (null) distribution
The new (treatment) distribution
The original (null) distribution

7. Effect Size
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
Hypothesis test tells us whether the observed difference is probably due to chance or not
It does not tell us how big the difference is
H0 is wrong
So there are two distributions
The new (treatment) distribution
The original (null) distribution
Effect size tells us how much the two populations don’t overlap

8. Effect Size Figuring effect size
But this is tied to the particular units of measurement
Figuring effect size
The new (treatment) distribution
The original (null) distribution
Effect size tells us how much the two populations don’t overlap

9. Effect Size Standardized effect size Cohen’s d
Puts into neutral units for comparison (same logic as z-scores)
Cohen’s d
The new (treatment) distribution
The original (null) distribution
Effect size tells us how much the two populations don’t overlap

10. Effect Size Effect size conventions small d = .2 medium d = .5
large d = .8
The new (treatment) distribution
The original (null) distribution
Effect size tells us how much the two populations don’t overlap

11. Error types There really isn’t an effect Real world (‘truth’)
H0 is correct
H0 is wrong
I conclude that there is an effect
Reject H0
Experimenter’s conclusions
I can’t detect an effect
Fail to Reject H0

12. Error types Real world (‘truth’) H0 is correct H0 is wrong
Type I error (): concluding that
there is a difference between groups
(“an effect”) when there really isn’t.
H0 is correct
H0 is wrong
Type I error
Reject H0
Experimenter’s conclusions
Type II error (): concluding that
there isn’t an effect, when there really is.
Fail to Reject H0
Type II error

13. Statistical Power
The probability of making a Type II error is related to Statistical Power
Statistical Power: The probability that the study will produce a statistically significant results if the research hypothesis is true (there is an effect)
So how do we compute this?

14. Statistical Power Real world (‘truth’) a = 0.05
H0: is true (is no treatment effect)
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
The original (null) distribution
a = 0.05
Reject H0
Fail to reject H0

15. Statistical Power Real world (‘truth’) a = 0.05
H0: is false (is a treatment effect)
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
The new (treatment) distribution
The original (null) distribution
a = 0.05
Fail to reject H0
Reject H0

16. Statistical Power Real world (‘truth’)
H0: is false (is a treatment effect)
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
The new (treatment) distribution
The original (null) distribution
b = probability of a Type II error
a = 0.05
Failing to
Reject H0, even
though there is
a treatment effect
Reject H0
Fail to reject H0

17. Statistical Power Real world (‘truth’)
H0: is false (is a treatment effect)
Real world (‘truth’)
H0 is correct
H0 is wrong
Type I error
Type II error
The new (treatment) distribution
The original (null) distribution
b = probability of a Type II error
a = 0.05
Power = 1 - b
Failing to
Reject H0, even
though there is
a treatment effect
Probability of
(correctly)
Rejecting H0
Reject H0
Fail to reject H0

18. Statistical Power Steps for figuring power
1) Gather the needed information: mean and standard deviation of the Null Population and the predicted mean of Treatment Population

19. Statistical Power Steps for figuring power
2) Figure the raw-score cutoff point on the comparison distribution to reject the null hypothesis
From the unit normal table: Z =
a = 0.05
Transform this z-score to a raw score

20. Statistical Power Steps for figuring power
3) Figure the Z score for this same point, but on the distribution of means for treatment Population
Remember to use the
properties of the
treatment population!
Transform this raw score to a z-score

21. Statistical Power Steps for figuring power
4) Use the normal curve table to figure the probability of getting a score more extreme than that Z score
b = probability of a Type II error
From the unit normal table: Z(0.355) =
Power = 1 - b
The probability of detecting this an effect of this size from these populations is 64%

22. Statistical Power Factors that affect Power: a-level Sample size
Population standard deviation 
Effect size
1-tail vs. 2-tailed

23. Statistical Power Factors that affect Power: a-level b Power = 1 - b
Change from a = 0.05 to 0.01
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

24. Statistical Power Factors that affect Power: a-level b Power = 1 - b
Change from a = 0.05 to 0.01
a = 0.05
a = 0.01 b
Power = 1 - b
Reject H0
Fail to reject H0

25. Statistical Power Factors that affect Power: a-level b Power = 1 - b
Change from a = 0.05 to 0.01
a = 0.05
a = 0.01 b
Power = 1 - b
Reject H0
Fail to reject H0

26. Statistical Power Factors that affect Power: a-level b Power = 1 - b
Change from a = 0.05 to 0.01
a = 0.05
a = 0.01 b
Power = 1 - b
Reject H0
Fail to reject H0

27. Statistical Power Factors that affect Power: a-level b Power = 1 - b
Change from a = 0.05 to 0.01
a = 0.05
a = 0.01 b
Power = 1 - b
Reject H0
Fail to reject H0

28. Statistical Power Factors that affect Power: a-level b Power = 1 - b
So as the a level gets
smaller, so does the
Power of the test
Change from a = 0.05 to 0.01
a = 0.05
a = 0.01 b
Power = 1 - b
Reject H0
Fail to reject H0

29. Statistical Power Factors that affect Power: Sample size b
Recall that sample size is related to the spread of the distribution
Change from n = 25 to 100
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

30. Statistical Power Factors that affect Power: Sample size b
Change from n = 25 to 100
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

31. Statistical Power Factors that affect Power: Sample size b
Change from n = 25 to 100
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

32. Statistical Power Factors that affect Power: Sample size b
Change from n = 25 to 100
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

33. Statistical Power Factors that affect Power: Sample size b
As the sample gets bigger, the standard error gets smaller and the Power gets larger
Change from n = 25 to 100
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

34. Statistical Power Factors that affect Power:
Population standard deviation
Change from  = 25 to 20
Recall that standard error is related to
the spread of the distribution
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

35. Statistical Power Factors that affect Power:
Population standard deviation
Change from  = 25 to 20
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

36. Statistical Power Factors that affect Power:
Population standard deviation
Change from  = 25 to 20
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

37. Statistical Power Factors that affect Power:
Population standard deviation
Change from  = 25 to 20
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

38. Statistical Power Factors that affect Power:
Population standard deviation
Change from  = 25 to 20
As the  gets smaller, the standard error
gets smaller and the Power gets larger
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

39. Statistical Power Factors that affect Power: Effect size mtreatment
Compare a small effect (difference) to a big effect
mtreatment
mno treatment
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

40. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0
mtreatment
mno treatment

41. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

42. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

43. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

44. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

45. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

46. Statistical Power Factors that affect Power: Effect size b
Compare a small effect (difference) to a big effect
a = 0.05
As the effect gets bigger, the Power gets larger b
Power = 1 - b
Reject H0
Fail to reject H0

47. Statistical Power Factors that affect Power: 1-tail vs. 2-tailed b
Change from a = 0.05 two-tailed to a = 0.05 two-tailed
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

48. Statistical Power Factors that affect Power: 1-tail vs. 2-tailed b
Change from a = 0.05 two-tailed to a = 0.05 two-tailed
p = 0.025
a = 0.05 b
Power = 1 - b
Reject H0
Fail to reject H0

49. Statistical Power Factors that affect Power: 1-tail vs. 2-tailed b
Change from a = 0.05 two-tailed to a = 0.05 two-tailed
a = 0.05
p = 0.025 b
Power = 1 - b
p = 0.025
Reject H0
Fail to reject H0

50. Statistical Power Factors that affect Power: 1-tail vs. 2-tailed b
Change from a = 0.05 two-tailed to a = 0.05 two-tailed
a = 0.05
p = 0.025 b
Power = 1 - b
p = 0.025
Reject H0
Fail to reject H0

51. Statistical Power Factors that affect Power: 1-tail vs. 2-tailed b
Change from a = 0.05 two-tailed to a = 0.05 two-tailed
a = 0.05
p = 0.025 b
Power = 1 - b
p = 0.025
Reject H0
Fail to reject H0

52. Statistical Power Factors that affect Power: 1-tail vs. 2-tailed b
Change from a = 0.05 two-tailed to a = 0.05 two-tailed
Two tailed functionally cuts the -level in half, which decreases the power.
a = 0.05
p = 0.025 b
Power = 1 - b
p = 0.025
Reject H0
Fail to reject H0

53. Statistical Power Factors that affect Power:
a-level: So as the a level gets smaller, so does the Power of the test
Sample size: As the sample gets bigger, the standard error gets smaller and the Power gets larger
Population standard deviation: As the population standard deviation gets smaller, the standard error gets smaller and the Power gets larger
Effect size: As the effect gets bigger, the Power gets larger
1-tail vs. 2-tailed: Two tailed functionally cuts the -level in half, which decreases the power

54. Why care about Power? Determining your sample size
Using an estimate of effect size, and population standard deviation, you can determine how many participants need to achieve a particular level of power
When a result if not statistically significant
Is is because there is no effect, or not enough power
When a result is significant
Statistical significance versus practical significance

55. Ways of Increasing Power