Psy B07 Chapter 8Slide 1 POWER

Psy B07 Chapter 8Slide 2 Chapter 4 flashback  Type I error is the probability of rejecting the null hypothesis when it is really true.  The probability of making a type I error is denoted as .

Psy B07 Chapter 8Slide 3 Chapter 4 flashback  Type II error is the probability of failing to reject a null hypothesis that is really false  The probability of making a type II error is denoted as .  In this chapter, you’ll often see these outcomes represented with distributions

Psy B07 Chapter 8Slide 4 Distributions  To make these representations clear, let’s first consider the situation where H 0 is, in fact, true:   Now assume that H0 is false (i.e., that some “treatment” has an effect on our dependent variable, shifting the mean to the right). correct failure to reject Alpha Type I Error

Psy B07 Chapter 8Slide 5 Distributions Distribution Under H 0 Correct Rejection Distribution Under H 1 Power Alpha Type II error

Psy B07 Chapter 8Slide 6 Definition of Power  Thus, power can be defined as follows:  Assuming some manipulation effects the dependent variable, power is the probability that the sample mean will be sufficiently different from the mean under H 0 to allow us to reject H 0.  As such, the power of an experiment depends on three (or four) factors:

Psy B07 Chapter 8Slide 7 Factors affecting power Alpha  As alpha is moved to the left (for example, if one used an alpha of 0.10 instead of 0.05), beta would decrease, power would increase... but, the probability of making a type I error would increase.  1 -  2 :  The further that H 1 is shifted away from H 0, the more power (and lower beta) an experiment will have.

Psy B07 Chapter 8Slide 8 Factors affecting power Standard error of the mean  The smaller the standard error of the mean (i.e., the less the two distributions overlap), the greater the power. As suggested by the CLT, the standard error of the mean is a function of the population variance and N. Thus, of all the factors mentioned, the only one we can really control is N.

Psy B07 Chapter 8Slide 9 Effect size  Most power calculations use a term called effect size which is actually a measure of the degree to which the H 0 and H 1 distributions overlap.  As such, effect size is sensitive to both the difference between the means under H 0 and H 1, and the standard deviation of the parent populations. Specifically:

Psy B07 Chapter 8Slide 10 Effect size  In English then, d is the number of standard deviations separating the mean of H 0 and the mean of H 1.  Note: N has not been incorporated in the above formula. You’ll see why shortly

Psy B07 Chapter 8Slide 11 Estimating effect size  As d forms the basis of all calculations of power, the first step in these calculations is to estimate d.  Since we do not typically know how big the effect will be a priori, we must make an educated guess on the basis of: 1) Prior research. 2) An assessment of the size of the effect that would be important. 3) General Rule (small effect d=0.2, medium effect d=0.5, large effect d = 0.8) effect d=0.5, large effect d = 0.8)

Psy B07 Chapter 8Slide 12 Estimating effect size  The calculation of d took into account 1) the difference between the means of H 0 and H 1 and 2) the standard deviation of the population.  However, it did not take into account the third variable the effects the overlap of the two distributions; N.

Psy B07 Chapter 8Slide 13 Estimating effect size  This was done purposefully so that we have one term that represents the relevant variables we, as experimenters, can do nothing about (d) and another representing the variable we can do something about; N.  The statistic we use to recombine these factors is called delta and is computed as follows:  where the specific ƒ(N) differs depending on the type of t-test you are computing the power for.

Psy B07 Chapter 8Slide 14 Power calcs for one-sample t  In the context of a one sample t-test, the ƒ (N) alluded to above is simply:  Thus, when calculating the power associated with a one sample t, you must go through the following steps: 1) Estimate d, or calculate it using:

Psy B07 Chapter 8Slide 15 Power calcs for one-sample t  Calculate δ using:  3) Go to the power table, and find the power associated with the calculated δ given the level of α you plan to use (or used) for the t-test

Psy B07 Chapter 8Slide 16 Power calcs for one-sample t Example: Say I find a new stats textbook and after looking at it, I think it will raise the average mark of the class by about 8 points. From previous classes, I am able to estimate the population standard deviation as 15. If I now test out the new text by using it with 20 new students, what is my power to reject the null hypothesis (that the new students marks are the same as the old students marks). How many new students would I have to test to bring my power up to.90? Note: Don’t worry about the bit on “noncentrality parameters” in the book.