Hypothesis Testing Hypothesis testing is an inferential process Using limited information to reach a general conclusion Observable evidence from the sample data Unobservable fact about the population Formulate a specific, testable research hypothesis about the population

Null Hypothesis no effect, no difference, no change, no relationship, no pattern, no … any pattern in the sample data is due to random sampling error

Type I Error A researcher finds evidence for a significant result when, in fact, there is no effect (no relationship) in the population. The researcher has, by chance, selected an extreme sample that appears to show the existence of an effect when there is none. The p-value identifies the probability of a Type I error.

Significance in Comparisons The key to becoming confident in a sample mean is determining the standard error of the mean. But if research was simply about becoming confident in and reporting our confidence in a sample mean, research would not be all that interesting.

Hillary C. Feeling Thermometer Gender Sample Mean Standard Error of the Sample Mean Squared Standard Error (Mean Variance) Female 55.7 (n=994, s=30.21) .96 .92 Male 46.4 (n=784, s=30.67) 1.10 1.20 TOTAL 51.2 (1778) Mean difference 9.3 Variance of the mean difference 2.12 Standard error of the mean difference 1.46 Here is the difference between the means in our sample, but how possible is it that the real difference in the population = 0? H0: The difference between the means in the population = 0.

Significance in Comparisons The key to figuring out whether the difference between two means is significant (different from 0 in the population), we need to calculate the standard error of the mean difference.

Std. Err. of the mean difference Hillary Clinton Feeling Thermometer Ratings, by Gender Gender Sample Mean Standard Error of the Sample Mean Squared Standard Error (Mean Variance) Female 55.7 (n=994, s=30.21) .96 .92 Male 46.4 (n=784, s=30.67) 1.10 1.20 TOTAL 51.2 (1778) Mean difference 9.3 Variance of the mean difference 2.12 Standard error of the mean difference 1.46 Step 1: Calculate these as before: s . Ö (n-1)

Std. Err. of the mean difference Hillary Clinton Feeling Thermometer Ratings, by Gender Gender Sample Mean Standard Error of the Sample Mean Squared Standard Error (Mean Variance) Female 55.7 (n=994, s=30.21) .96 .92 Male 46.4 (n=784, s=30.67) 1.10 1.20 TOTAL 51.2 (1778) Mean difference 9.3 Variance of the mean difference 2.12 Standard error of the mean difference 1.46 Step 2: Square the standard errors

Std. Err. of the mean difference Hillary Clinton Feeling Thermometer Ratings, by Gender Gender Sample Mean Standard Error of the Sample Mean Squared Standard Error (Mean Variance) Female 55.7 (n=994, s=30.21) .96 .92 Male 46.4 (n=784, s=30.67) 1.10 1.20 TOTAL 51.2 (1778) Mean difference 9.3 Variance of the mean difference 2.12 Standard error of the mean difference 1.46 Step 3: Sum the squared standard errors

Std. Err. of the mean difference Hillary Clinton Feeling Thermometer Ratings, by Gender Gender Sample Mean Standard Error of the Sample Mean Squared Standard Error (Mean Variance) Female 55.7 (n=994, s=30.21) .96 .92 Male 46.4 (n=784, s=30.67) 1.10 1.20 TOTAL 51.2 (1778) Mean difference 9.3 Variance of the mean difference 2.12 Standard error of the mean difference 1.46 Step 4: Take the square root of the variance of the mean difference

95% confidence interval Armed with the standard error of the mean difference, we can proceed with the familiar process of setting up our 95% confidence interval: 95% c.i. = mean difference + (standard error of mean difference)(critical t-score) 95% c.i. = 9.3 + (1.46)(1.96) = 6.44 < mean difference in population < 12.16

Significance in Comparisons H0: Difference in means in the population = 0. H1: Difference in means in the population > 0. We are 95% confident that the difference in the means in the population falls somewhere between 6.44 and 12.16. Therefore, we can reject H0.

But we should be more precise. There is a 2 But we should be more precise. There is a 2.5% chance that the difference between the means in the population is < 6.44, and 2.5% chance that the difference between the means in the population is > 12.16. We don’t really care what chance there is that the difference between the means is > 12.16. We only care that it is > 0. We don’t care about the results in this tail. 2.5% of results in this tail 2.5% of results in this tail 34% 34% 47.5% 47.5% m 1.96 s.d. 1 s.d. 1 s.d. 1.96 s.d.

Two-tailed tests of significance: used to test alternative hypotheses that state that something is significantly different from zero. One-tailed tests of significance: used to test alternative hypotheses that state that something is significantly greater than or less than zero. H0: Difference in means in the population = 0. H1: Difference in means in the population > 0. We don’t really care what chance there is that the difference between the means is > 12.16. We only care that it is > 0. So we take all of the critical area under the curve from here. . 2.5% of results in this tail 2.5% of results in this tail 34% 34% 47.5% 47.5% m 1.96 s.d. 1 s.d. 1 s.d. 1.96 s.d.

5% of values under the curve Two-tailed tests of significance: used to test alternative hypotheses that state that something is significantly different from zero. One-tailed tests of significance: used to test alternative hypotheses that state that something is significantly greater than or less than zero. H0: Difference in means in the population = 0. H1: Difference in means in the population > 0. We don’t really care what chance there is that the difference between the means is > 12.16. We only care that it is > 0. So we take all of the critical area under the curve from here. . And add it here. . 2.5% of results in this tail 34% 34% 47.5% 47.5% m 1.96 s.d. 1 s.d. 1 s.d. 1.96 s.d. 5% of values under the curve

5% of values under the curve One-tailed tests of significance: used to test alternative hypotheses that state that something is significantly greater than or less than zero. H0: Difference in means in the population = 0. H1: Difference in means in the population > 0. “What is the probability that the population difference could be as low as 0? Is this probability less than .05?” mean difference - (standard error of mean difference)(critical t-score) 9.3 – (1.46)(1.645) = 6.9 The lowest plausible value for the difference in the population is 6.9, NOT 0. Therefore, H0 is rejected. Now find the critical t-score by determining your degrees of freedom, then looking at the .05 column (rather than the .025 column). 34% 34% 47.5% 47.5% m 1.96 s.d. 1 s.d. 1 s.d. 1.96 s.d. 5% of values under the curve

Maximum Precision: Test Statistics and P-values In our example, instead of saying “there is a less than 5% chance that the mean differences in the population = 0”, we can be even more precise. We might ask: “What is the exact probability that the mean difference = 0?” For this, we use test statistics. From our example, to generate a test statistic on the difference of means, we need 3 components: The difference associated with the alternative hypothesis, H1 (which we observed in the sample) = 9.3 The difference claimed by the null hypothesis, H0 = 0 The standard error of the difference = 1.46 This is a rule of thumb: take any sample statistic and divide it by its standard error. If the result > 2, the coefficient is significant. t = (H1 – H0) / standard error of the difference t = (9.3 – 0) / (1.46) t = 6.37 p = .00000000012088 Every t-score has an associated probability value, or p-value. Most statistical packages cut of the measure at .000 (one in ten-thousand).

Every other step is exactly the same. To find the standard error of the difference of proportions, we proceed exactly as before, with only one difference—calculating the initial standard error of the sample proportions. s = Ö (p)(q) Ö (n-1) Every other step is exactly the same. --Proportions Favoring Gay Adoptions, by Gender Gender Sample Proportion Standard Error of the Sample Proportion Variance of Sample Proportion Female .497 (n=922) .016 .00027 Male .394 (n=734) .018 .00032 Difference in proportions .103 Variance of the difference .00059 Standard error of the difference .0243