Copyright © 2011 by Pearson Education, Inc. All rights reserved Statistics for the Behavioral and Social Sciences: A Brief Course Fifth Edition Arthur Aron, Elaine N. Aron, Elliot Coups Prepared by: Genna Hymowitz Stony Brook University This multimedia product and its contents are protected under copyright law. The following are prohibited by law: -any public performance or display, including transmission of any image over a network; -preparation of any derivative work, including the extraction, in whole or in part, of any images; -any rental, lease, or lending of the program.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Introduction to Hypothesis Testing Chapter 5

Copyright © 2011 by Pearson Education, Inc. All rights reserved Chapter Outline A Hypothesis-Testing Example The Core Logic of Hypothesis Testing The Hypothesis-Testing Process One-Tailed and Two-Tailed Hypothesis Tests Decision Errors Hypothesis Tests as Reported in Research Articles

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing A systematic procedure for deciding whether the results of a research study supports a hypothesis that applies to a population –hypothesis a prediction intended to be tested in a research study can be based on informal observation or theory –theory a set of principles that attempts to explain one or more facts, relationships, or events usually gives rise to various specific hypotheses that can be tested in research studies

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing Researchers want to draw conclusions about a particular population. –e.g., babies in general Conclusions will be based on results of studying a sample. –e.g., one baby

Copyright © 2011 by Pearson Education, Inc. All rights reserved The Core Logic of Hypothesis Testing Researchers must spell out in advance what would have to happen in order to allow them to conclude that their hypothesis was supported. They then conduct their experiment. Then they figure the probability of getting their particular experimental result if their hypothesis was not true. –They answer the question: What is the probability of getting our research results if the opposite of what is predicted were true? –If it is highly unlikely that we would get our research results if the opposite of what we are predicting were true: »We can reject the opposite prediction. »If we reject the opposite prediction, we can accept our prediction.

Copyright © 2011 by Pearson Education, Inc. All rights reserved The Hypothesis-Testing Process Step 1: Restate the question as a research hypothesis and a null hypothesis about the population. Step 2: Determine the characteristics of the comparison distribution. Step 3: Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected. Step 4: Determine your sample’s score on the comparison distribution. Step 5: Decide whether to reject the null hypothesis.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing: Step 1 Restate the question as a research hypothesis and a null hypothesis about the populations. –research hypothesis statement in hypothesis testing about the predicted relationship between populations –null hypothesis statement about a relationship between populations that is the opposite of the research hypothesis –examples Population 1: students who participate in a reading program Population 2: students in general (who do not participate in the reading program) –research hypothesis »Students who participate in this reading program (Population 1) will be able to read at a higher level than students in general (Population 2). –null hypothesis »There is no difference in reading level between students in the reading program and students not in the reading program..

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing: Step 2 Determine the characteristics of the comparison distribution. –comparison distribution distribution used in hypothesis testing –represents the population distribution if the null hypothesis is true »distribution to which you compare the score based on your sample’s results –Find out the key information about the comparison distribution. e.g., population mean, population SD, shape of the distribution (does it follow a normal curve?) –If the null hypothesis is true: Population 1 and Population 2 are the same.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing: Step 3 Set a cutoff sample score or critical value. –This is a target against which you will compare the results of your study. By setting a cutoff score, you are deciding how extreme a sample score would need to be in order to be too unlikely to get such an extreme score if the null hypothesis were true. –Researchers use Z scores and percentages to set the cutoff scores. »For instance, a researcher might decide that if a result was less likely than 1%, she would reject the null hypothesis. »In this case, researchers would look at the normal curve table and find the Z score cutoff for scores in the top 1% of a normal curve, which is »Generally, researchers in the social and behavioral sciences use conventional levels of significance, which are cutoff scores of either 5% or 1%. »When a sample score is at least as extreme as the cutoff score, then the result is considered statistically significant.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing: Step 4 Determine your samples score on the comparison distribution. Figure the Z score for the sample’s raw score based on the comparison distribution’s mean and standard deviation (the population mean and the population standard deviation). If your sample’s raw score = 20, the population mean = 10, and the population standard deviation = 2 The Z score for your sample would be (20 – 10) / 2 = 5

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Testing: Step 5 Decide Whether to Accept or Reject the Null Hypothesis. –Compare your sample’s Z score to the cutoff Z score. If we had set the cutoff score at 1% (Z score of 2.33) and the Z score of our sample was 5, we would be able to reject the null hypothesis that there is no difference between our sample and the comparison population because our sample’s Z score is more extreme then the cutoff Z score.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Implications of Rejecting of Failing to Reject the Null Hypothesis When you reject the null hypothesis, all you are saying is that your results support the research hypothesis. –The results never prove the research hypothesis or show that your hypothesis is true. Research studies and their results are based on the probability or chance of getting your result if the null hypothesis were true. When the results are not extreme enough to reject the null hypothesis, you do not say that the results support the null hypothesis. –You say that the results are not statistically significant, or that the results are inconclusive. We are basing research on probabilities, and the fact that we did not find a result in this study does not mean that the null hypothesis is true.

Copyright © 2011 by Pearson Education, Inc. All rights reserved One-Tailed and Two-Tailed Hypothesis Tests Directional Hypothesis –focuses on a specific direction of effect e.g., that reading levels would be greater in students participating in a reading program –one-tailed test To reject the null hypothesis, a sample score needs to be in a particular tail of the distribution (e.g., the top 1% of the distribution). Non-Directional Hypothesis –a hypothesis that predicts an effect, but does not specify whether the score will be high or low –The null hypothesis would be that there would be no change, or that the scores would not be extreme at either tail of the comparison distribution.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Determining Cutoff Scores with Two-Tailed Tests For a two-tailed test, you have to divide the significance percentage between two tails. –For a 5% significance level, the null hypothesis would be rejected if the sample score was in either the top 2.5% or the bottom 2.5% of the comparison distribution.

Copyright © 2011 by Pearson Education, Inc. All rights reserved When to Use One-Tailed or Two-Tailed Tests Use a one-tailed test when you have a clearly directional hypothesis. Use a two-tailed test when you have a clearly non-directional hypothesis. With a one-tailed test, if the sample score is extreme—but in the opposite direction—the null cannot be rejected. Often researchers will use two-tailed tests even if the hypothesis is directional.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Decision Errors When the right procedures lead to the wrong decisions In spite of calculating everything correctly, conclusions drawn from hypothesis testing can still be incorrect. This is possible because you are making decisions about populations based on information in samples. –Hypothesis testing is based on probability.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Type I Error Rejecting the null hypothesis when the null hypothesis is true –You find an effect when in fact there is no effect. A Type I error is a serious error as theories, research programs, treatment programs, and social programs are often based on conclusions of research studies. The chance of making a Type I error is the same as the significance level. –If the significance level was set at p <.01, there is less than a 1% chance that you could have gotten your result if the null hypothesis was true. –To reduce the chance of making a Type I error, researchers can set a very stringent significance level (e.g., p <.001).

Copyright © 2011 by Pearson Education, Inc. All rights reserved Type II Error With a very extreme significance level, there is a greater probability that you will not reject the null hypothesis when the research hypothesis is actually true. –concluding that there is no effect when there is actually an effect The probability of making a Type II error can be reduced by setting a very lenient significance level (e.g., p <.10).

Copyright © 2011 by Pearson Education, Inc. All rights reserved Relationship Between Type I and Type II Errors Decreasing the probability of a Type I error increases the probability of a Type II error. –The compromise is to use standard significance levels of p <.05 and p <.01.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Hypothesis Tests as Reported in Research Articles In research articles, for each result of interest, the researcher usually says whether the result was statistically significant. The researcher gives the symbol for the specific method used in figuring out the probabilities. There will be an indication of significance level (e.g., p <.05 or p <.01). Usually a two-tailed test is used; if this is not the case, the researcher will generally specify that a one-tailed test was used.

Copyright © 2011 by Pearson Education, Inc. All rights reserved Key Points Hypothesis testing considers the probability that the results of a study could have come about even if the experimental procedure had no effect. If this probability is low, the scenario of no effect is rejected and the hypothesis behind the experimental procedure is supported. The expectation of an effect is the research hypothesis; the hypothetical situation of no effect is the null hypothesis. When a result is so extreme that it would be very unlikely if the null hypothesis were true, the null hypothesis is rejected and the research hypothesis is supported. If the result is not that extreme, the researcher does not reject the null hypothesis and the study is inconclusive. Behavioral and social scientists usually consider a result extreme enough if it is less likely than 5% that you would get that result if the null hypothesis was true. A significance level of 1% is also commonly used in research. The cutoff percentage is the probability of the result being extreme in a predicted direction in a directional or one-tailed test. The cutoff percentages are the probability of the result being extreme in either direction in a nondirectional or two-tailed test. Steps for hypothesis testing are: –Restate the question as a research hypothesis and a null hypothesis about the population. –Determine the characteristics of the comparison distribution. –Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected. –Determine your sample’s score on the comparison distribution. –Decide whether to reject the null hypothesis. A Type I error is when a researcher rejects the null hypothesis but the null hypothesis is actually true. A Type II error is when a researcher does not reject the null hypothesis, but the null hypothesis is actually false. Research articles report the results of hypothesis testing by saying whether the results were significant, giving the cutoff sample score on the comparison distribution, and giving the probability level of the cutoff sample score.