Chapter 8: Introduction to Hypothesis Testing

2 Hypothesis Testing An inferential procedure that uses sample data to evaluate the credibility of a hypothesis about a population.

3 Before and after treatment comparisons µ = 18 µ = ?  = 4 Known population before treatment Unknown population after treatment TreatmentTreatment

4 Hypothesis Testing Step 1: State hypothesis Step 2: Set criteria for decision Step 3: Collect sample data Step 4: Evaluate null hypothesis Step 5: Conclusion

5 Infant handling problem We know that the average GRE score for college seniors is µ = 500 and  = 100. A researcher is interested in effect of the Kaplan course on GRE scores. A random sample of 100 college seniors is selected and take the Kaplan GRE Training course. Afterwards, each is given the GRE exam. Average scores after training are 525 for the sample of 100 students.

6 Before and after treatment comparisons µ = 500 µ = ?  =100 Known population before treatment Unknown population after treatment TreatmentTreatment

7 Hypothesis testing example problem µ = 500 g  = 100 g Step 1: Ho:Ho: µ = 500 GRE after Kaplan Course GRE after Kaplan Course ( There is no effect of the Kaplan training course on average GRE scores ) H1:H1: µ  500 (There is an effect…)  = 0.05 Step 2: Set criteria z Critical Region z > 1.96 z < or Step 3:n = 100X = 525 Step 4: Reject H o because Z obt of 2.5 is in the critical region. Step 5: Conclusion. The Kaplan training course significantly increased GRE scores on average, z = 2.5, p <.05.

8 Evaluating Hypotheses Reject H o Middle 95% Very Unlikely X µ z

What is the effect of alcohol on fetal development? 9

10 Before and after treatment comparisons µ = 18 µ = ?  = 4 Known population before treatment Unknown population after treatment TreatmentTreatment

11 Hypothesis testing example problem µ = 18 g  = 4 g Step 1: Ho:Ho:µ = 18 g Weight of rats of alcoholic mothers (There is no effect of alcohol on the average birth weight of new born rat pups) H1:H1: µ  18 g (There is an effect…)  = 0.01 Step 2: Set criteria z Critical Region z > 2.58 z < or Step 3:n = 25 ratsX = 15.5 g Step 4: Reject H o because Z obt of is in the critical region. Step 5: Conclusion: Alcohol significantly reduced mean birth weight of newborn rat pups, z = 3.125, p <.01.

12 Evaluating Hypotheses Reject H o Middle 99% Very Unlikely X µ z

13 The distribution of sample means (all possible experimental outcomes) if the null hypothesis is true Reject H o Extreme values (probability < alpha) if H o is true Possible, but “very unlikely”, outcomes

14 Normal curve with different alpha levels z  =.05  =.01  =.001

15 Reject H o Null Hypothesis Rejection areas on distribution curve p >  p < 

16 Hypothesis testing example problem What if the sample mean does not indicate a large enough change to reject the null? Step 1: Ho:Ho: µ = 500 GRE after Kaplan Course GRE after Kaplan Course ( There is no effect of the Kaplan training course on average GRE scores ) H1:H1: µ  500 (There is an effect…)  = 0.05 Step 2: Set criteria z Critical Region z > 1.96 z < or Step 3:n = 100X = 515 Step 4: Retain H o because Z obt of 1.5 is not in the critical region. Step 5: Conclusion. There was no effect of the Kaplan training on GRE scores on average, z = 1.5, p >.05.

17 Hypothesis testing example problem What if the sample mean is not in the critical region? Step 1: Ho:Ho:µ = 18 g Weight of rats of alcoholic mothers (There is no effect of alcohol on the average birth weight of new born rat pups) H1:H1: µ  18 g (There is an effect…)  = 0.01 Step 2: Set criteria z Critical Region z > 2.58 z < or Step 3:n = 25 ratsX = 17g Step 4: Retain H o because Z obt of is not in the critical region. Step 5: Conclusion: There was no effect of alcohol on the average birth weight of newborn rat pups, z = 1.25, p >.01.

18 Hypothesis testing procedure 1.State hypothesis and set alpha level 2.Locate critical region e.g.z > | 1.96 | z > 1.96 or z < Obtain sample data and compute test statistic 4. Make a decision about the H o 5. State the conclusion

19 Error Type / Correct Decision table Type I Error Type II Error Decision Correct Reject H o Retain H o Experimenter’s Decision Actual Situation No Effect, H o True Effect Exists, H o False

20 Jury’s Verdict vs. Actual Situation Type I Error Type II Error Verdict Correct Guilty Innocent Jury’s Verdict Actual Situation Did Not Commit Crime Committed Crime

21 Fixed coin (cheating) vs. OK coin (fair) Type I Error Type II Error Correct Decision Coin Fixed (Cheating) Coin O.K. (Fair) Your Decision Actual Situation Coin O.K. (Fair) Coin Fixed (Cheating)

22 Assumptions for Hypothesis Tests with z-scores: 1.Random sampling 2.Value of  unchanged by treatment 3.Sampling distribution normal 4.Independent observations

One-Tailed Hypothesis Tests 23

24 Reading Improvement Problem A researcher wants to assess the “miraculous” claims of improvement made in a TV ad about a phonetic reading instruction program or package. We know that scores on a standardized reading test for 9-year olds form a normal distribution with µ = 45 and  = 10. A random sample of n = 25 8-year olds is given the reading program package for a year. At age 9, the sample is given the standardized reading test.

25 Normal curve with rejection area on the positive side Reject H o Data indicates that H o is wrong 45 µ z X

26 Two-tailed vs. One-tailed Tests 1.In general, use two-tailed test 2.Journals generally require two-tailed tests 3.Testing H o not H 1 4.Downside of one-tailed tests: what if you get a large effect in the unpredicted direction? Must retain the H o

27 Error and Power Type I error =  –Probability of a false alarm Type II error =  –Probability of missing an effect when H o is really false Power = 1 -  –Probability of correctly detecting effect when H o is really false

28 Factors Influencing Power (1 -  ) 1.Sample size 2.One-tailed versus two-tailed test 3.Criterion (  level) 4.Size of treatment effect 5.Design of study

29 Distributions demonstrating power at the.05 and.01 levels  Treatment Distribution Null Distribution  =.05 Reject H o X  Treatment Distribution Null Distribution  =.01 Reject H o X 1 -  Reject H o Reject H o 1 -  µ = 180 µ = 200 (a) (b)

30 Distributions demonstrating power with different size n’s  Treatment Distribution Null Distribution n = 25 Reject H o X 1 -  Reject H o µ = 180µ = 200 (a)  Treatment Distribution Null Distribution n = 100 Reject H o X 1 -  (b) Reject H o

31 Large distribution demonstrating power  Treatment Distribution Null Distribution Reject H o X 1 -  Reject H o µ 180 µ 200

32 Large distribution demonstrating power (with closer means)  Treatment Distribution Null Distribution Reject H o X 1 -  Reject H o µ 190 µ 200

33 Frequency distributions Frequency Frequency X X

34 Are birth weights for babies of mothers who smoked during pregnancy significantly different? µ = 2.9 kg  = 2.9 kg Random Sample:n = , 2.0, 2.2, 2.8, 3.2, 2.2, 2.5, 2.4, 2.4, 2.1, 2.3, 2.6, 2.0, 2.3

35 The distribution of sample means if the null hypothesis is true (all the possible outcomes) Sample means close to H o : high-probability values if H o is true µ from H o Extreme, low-probability values if H o is true Extreme, low-probability values if H o is true

36 Distribution of sample means based on percentages Middle 95%: high-probability values if H o is true µ from H o Reject H o z = z = 1.96 Critical Region: Extreme 5%